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ABSTRACT 

This report examines student performance on a test 
composed of items developed to be aligned with the Charleston County 
School District (CCSD) Language Arts Comprehension Curriculum. The 
report contaiins a data analysis section that explains the 
relationship tetween two basic concepts of the Rasch measurement 
model, item difficulty and student ability. The report then discusses 
curriculum charts which have been constructed for grades 1 through 5. 
Item difficulty, student ability, and item response patterns were 
used to analyze group performance on the curriculum objectives within 
each of the 14 domains of the CCSD Language Arts Comprehension 
Curriculum. The domains were collapsed into five content areas: 
vocabulary knowledge; main idea; literal comprehension; inferential 
comprehension; and analysis of literature. The report includes 
recommendations within each content area. Ten figures aire also 
included. (KS) 
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Executive Stimmary 

This report examines student performance on a test composed 
of items developed to be aligned with the CCSD Language Arts 
Comprehension Curriculum. Test items were pilot tested in May of 
1986, and analyzed using the Rasch measurement model. This 
report is divided into three sections: Data Analysis, Curriculum 
Charts, and Analysis by Objectives. 

The Data Analysis section explains the relationship between* 
two basic concepts of the Rasch measurement model. These con- 
cepts are *oem difficulty and stud ent abi.l i tv. Item difficulty 
and student ability are each measured with the same scale and can 
be plotted along the same continuum. Two caveats are stressed. 
First, student ability as used in this report should not be 
confused with not ioas .such as general ability. Rather,, student 
ability refers to the Rasch value that locates a person along an 
achievement dimension, and is represented by a student's raw 
score on a test. Second, one must be careful about interpreting 
as "complete mastery" a situation in which student ability is 
greater than the difficulty of items testing a domain of knowledge 

Curriculum charts were constructed for grades 1 through 5. 
Curriculum charts are graphic representations of the structure of 
a set of curriculum objectives. A curriculum chart is 
constructed by averaging the difficulties of items assessing each 
curriculum objective and plotting these averages on a graph with 
difficulty along the ordinate and curriculum domains along the 
abscissa. Since item difficulty and student ability are on the 
same dimension, average student ability for each grade level can 



be represBnted by a line crossing the ordinate at the appro- 
priate point. For comprehension, the average student ability was 
greater than the difficulty of most objectives in this curricu- 
lum. This means that, on the average, students have learned most 
of the material specified in the curriculum. This result is 
expected since the items were pilot tested at the end of the 
school year. Several uses for curriculum charts were delineated in 
this section of the report. 

Item difficulty, student ability, and item response patterns 
were used to analyze group performance on the curriculum objec- 
tives within each of the fourteen domains of the CCSD Language 
Arts Comprehension Curriculum, The domains were collapsed into 
five content areas that are closely aligned to the South Carolina 
Basic Skills Assessment Program. These content areas are Vocabu- 
lary .Knowledge.,- Main Idea, Literal Comprehension, Inferential 
Comprehension, and Analysis of Literature. In addition to per- 
formance results, this section contains general recommendations 
for instruction in each of the five broad content areas. High- 
lights of this section are as foll::ws: 

Vocabu l ary knowledge . Analysis of performance on the voca- 
bulary items indicated that in grades 3 through 5 children had 
the most difficulty with structural word meaning and contextual 
word meaning items. Multiple meaning items were difficult when 
the word was used in an unusual way or when the context was not 
"rich." Student performance on the analogy items suggested that 
vocabulary development is very idiosyncratic. Recommendations 
for instruction focused on activities that underscore the 
development of both precision and flexibility in vocabulary knowledge. 
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Main idea. In grades 1 and 2 students had difficulty recog- 
nizing a main idea statement that was not a direct restatement of 
a main idea sentence from the reading selection. Recognizing a 
paraphrased main idea statement was as difficult as recognizing 
an inferred statement of a main idea, until grade 3, when recog- 
nizing restated and paraphrased main idea statements became of 
equal and less difficulty than recognizing an inferred main idea. 
Analysis of incorrect response patterns indicated that by grade 5 
students were not respr »dinq randomly. Instead, they tended to 
choose statements too narrow or too broad to be the main idea. 
Instructional activities incorporating verbal modeling of 
processes, and graphic organizers were recommended. 

Li tera 1 compr ehgns i on . Data from performance on literal 
comprehension items suggested degrees of instructional emphasis 
that could be placed on particular objectives at the various 
grade levels. The difficulty of detail items dropped off from 
grade 1 to grade 2. It was recommended that teachers spend less 
instruct ional time on de tails in grade II , and place greater 
emphasis on other literal comprehension skills. The difficulty 
level of detail items increased again at grade 3, perhaps 
reflecting the expansion in the breadth of text children are 
beginning to encounter. It was suggested that the instructional 
emphasis on details should increase, with a focus on distin- 
guishing between relevant and irrelevant details. By grades 4 
and 5 the difficulty level of detail items once again dropped 
off: it was recommended that instructional emphasis reflect this 
drop with a shift to other, more difficult objectives in the 
curriculum. 
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2/" Inferential comprehension . Difficulty of inferential com- 

^ I. 

' prehension items was related to the number of sentences from 

which the student had to draw information to make the inference, 
[. the amount and clarity of available cues for making the 

I' inference, and the extent to which the student had to draw upon 

' his/her own fxand of knowledge about the topic. Recommended 

instructional activities emphasized teacher modeling of problem 
F solving strategies and group prereading activities designed to 

1 activate prior knowledge about a topic. 

[ Analysis o f 1 i terature . Analysis of literature items tested 

the students* abilities to critically examine written text, 
r Specific skills in the areas of figurative language, fantasy/ 

• reality, fact /opinion , and structural elements were assessed. 

5. Among figurative language items those containing metaphors and 

-personification were the most difficult. Students had more dif- 
ficulty identifying fact statements than opinion statements. In 
grades 2 through 4 students seemed to be cued to incorrect answer 
responses that contained language used in the stimulus. Recom- 
mended instructional activities included the use of fables for 
teaching the difference between fantasy and reality: teaching the 
structure of fact and opinion statements: and the use of story 
maps as a device for helping students develop a schema for 
narrative text. 




Data Analysis 

« 

This report examines student performance on a test composed 
of items developed to be aligned with the CCSD Language Arts 
Comprehension Curriculum* The test items were pilot tested in 
May of 1986 and analyzed using the Rasch measurement model. 

In this report reference is frequently made to two test 
statistics. These statistics are i t em rij f f icultY and student 
abi 1 i ty . Both statistics are on the same scale which is divided 
into units of measure called lo qi ts . 

Rasch analysis results in the assignment of a value in logit 
units to each test item. This value represents the itetn's diffi- 
culty level relative to other items in the item bank. The pro- 
cess of assigning h value of difficulty to each test item is 
called item cal ibrat j.on. An item's difficulty increases in pro- 
portion to the number of incorrect responses made by students to 
the item. Calibrated items can be placed along a continuum from 
easy to hard. 



Test Items 

1 2 3 4 5 6 78 9 10 
Easy Items Hard Items 

Item Calibrations 

This approach was used to create language arts item banks 
for grades 1 through 8. In addition, the language arts items 
were vertically e quat ed into one overall bank. Following wi thin- 
grade calibration, the item difficulties were linked together on 
a common dimension of difficulty. Vertical equating allowed for 
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comparisons between the difficulty of objectives and the ability 
of students across grade level s . 

The purpose of **testing" is to estimate a person's location 
along an achievement dimension implied by a test. The term st^udent 
abi 1 i ty is used to represent the Rasch value that locates the 
person along this achievement dimension. A student ' s abi 1 i ty 
increases in proportion to the number of his/her correct 
responses on ohe test. 

Since item difficulties and student abilities are measured 
with the same units (i.e.. logits) they can each be plotted along 
the same continuum. Below is an example of the performance of 
two persons on a ten item test. 



Low Abi 1 i ty 
A 



Persons 
B 



High Abi 1 i ty 



Test Items 



1 23456789 10 
Easy Items Hard Items 

I tem Cal ibrat ions 



To understand the concept of student ability it is important 
that one not confuse it with notions such as general ability 
(i.e., "g''), or aptitude. Rather, one should think of student 
ability in terms of the probability of giving a correct response 
to a test item. When the ability level of a person is greater 
than the difficulty level of an item, then the probability of the 
person answering the item correctly is greater than one half. 
However, when the ability level of a person is less than the 
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difficulty level of an item, the probability of the person 
answering the question correctly is less than one half. Finally, 
when levels of ability and difficulty are the same, the proba- 
bility of the person answering the question correctly is exactly 
one half. Refer to the figure above. The probability of person 
B answering item 1 correctly is greater than one half, while the 
probability of person B answering item 10 correctly is less than 
one half. Thus, it is likely that person B can answer item 1 
correctly but urUikely that s/he will be able to answer item 10 
correctly. The probability of person B answering test item 6 
correctly is one half; that is, person B has a 50-50 chance of 
an<5wering item 6 correctly. 

When comparing item difficulty and student ability, another 
caveat must be stressed. *If a student's ability were higher than 
the difficulty level of all of a set of items testing a domain of 
knowledge, one might be tempted to interpret this as •'complete 
mastery** of the domain, particularly if the student were to 
obtain a perfect test score. But unless the set of items actual- 
ly contained the most difficult item that could ever be written for 
that domain there would always be the possibility of other items 
which were even more difficult. On the hardest possible test 
even a "perfectly scoring student'* might answer items incor- 
rectly, revealing that mastery was not complete at all. In the 
case of language comprehension, a more accurate view of the 
comparison between student ability and item difficulty is that it 
illustrates progress through a curriculum. 
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A curriculum chart is a graphic representation of the struc- 
ture of a set of curriculum objectives. A curriculum chart is 
constructed by averaging the difficulties of items assessing each 
curriculum objective and plotting these values on a graph with 
difficulty level along the ordinate* and the curriculum domains 
along the abscissa. Since item difficulty and student ability 
are on the same dimension, the average student abi 1 i ty for each 
grade level can be represented by a line crossing the ordinate at 
the appropriate point. In this way average student ability can 
be used as an estimate of the typical student*s standing relative 
to the curriculum objectives. 

Curricul'om charts of the Language Arts Comprehension Curricu- 
lum h^ve been developed for grades 1 throuL,n 5. (Refer to. 
Figures 1 through 5 followiri this section.) The ability lines 
show that, on the average, student ability is greater than item 
difficulty, indicating that the average student has learned most 
of the material specified in the curriculum. These high levels 
of achievement are to be expected since the items were pilot 
tasted at the end of the school year. They do not mean the pilot 
test was too easy for the students. 

Curr iculum charts have sever a 1 uses . They i 1 lustra te the 
relative difficulty of the objectives in a curriculum. Teachers 
can use this information to plan instructional time, or to 
ence objectives. That is, teachers can allocate more 
ct ions 1 t ima to the more difficult objectives: and /or 
.-rs can teach the easier objectives earlier in the year. 
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Raw scores can be converted into an ability measure for each 
student • These student abilities can be plotted on the curricu- 
lum chart. Teachers can see where an entire class or one indivi- 
dual falls along the curriculum continuum. This information 
could be used to form instructional groups, or combined with 
right-response data to plan remedial instruction for one indivi- 
dual « 

Curriculum charts can be used as growth charts. An indivi- 
dual's ability measure can be plotted on the chart at designated 
times during the school year or over grade levels to illustrate 
progress through the curriculum. Such a report might be 
informative to parents. 
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Figure I. Curriculum chart for grade 1. 
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Figure 2. Curriculum chart for grade 2. 
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Figure 3. Curriculum chart for grade 3. 



16 



17 



more 
difficult 



Difficulty 



less 
difficult 



-2 



10.3 conclusions 



^9.0 cause/effect 



Avg. Student 
Ability =1.7 



♦ 14.0 slructuial 
elements 

^5 



13.2 fact/opinion 



8.0 compare/contrast 

^ 5.0 target # A 

^ 1.2 synonymf; ^3.3 sequence ^\ 

^,^^-—1.0 target #11.3 outcomes 12.O figurative 

^1.1 antonyms Z directions language 

^ 7.2 analogy 
*6.0 classify 



^1.3 homonyms 



#2.4 details 



1 

Vocabulary 
Knowledge 



r- 

Main 
idea 



+ 



Literal Inferential 
• Content Areas 



+ 



Analysis of 
Literature 



Figure 4. Curriculum chart for grade 4. 
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Fiqnxe 5. Curriculum chart for grade 5, 



Analysis by Objectives 



Item difficulty, student ability, and item response pat.terns 
were used to analyze group performance on objectives of the 
Language Arts Comprehension Curriculum. At the time the items 
test ing these ob ject i ves were pi loled Charleston County' s 
Language Arts Comprehension Curriculum consisted of fourteen 
domains representing literal, inferential, and evaluative skills. 
Within each domain were a set of hierarchically arranged 
objectives, culminating with a •^target" objective. The 
hierarchical arrangement was designed to define an instructional 
program for each domain. 

The curriculum has since been, restructured, however, the 
revisions were not available at the time this report was being 
prepared. For this report the fourteen domains were collapsed 
into five content areas that are closely aligned to the South 
Carolina Basic Skills Assessment Program. These content areas 
are Vocabulary Knowledge, Main Idea, Literal Comprehension, 
Inferential Comprehension, and Analysis of Literature. 

I nterpretat 1 ons of ob jec t ive or doma in difficulty were made 
relative to student ability. As the difference between average 
student ability and average item difficulty increased the 
difficulty level of the objective or domain was said to decrease. 
Similarly, as the difference between the two decreased, the 
difficulty level of an objective or domain was said to increase. 
This interpretation is in accordance with the concept of the 
probability of giving a correct response to a test item (cf the 
Data Analysis section of this report). 



Vocabulary Knowledge 



Three domains (Word Meaning, Classification, and Analogies) 
have been grouped together under one content area. Vocabulary 
Knowledge. Ability to answer these test items appeared to be 
more dependent upon understanding the meanings of individual 
words than upon understanding the meaning of connected text. 
Figure 6 shows the relative difficulty among vocabulary knowledge 
ob ject ives in grades 1 through 5 . 

Domain 1 : Word Meaning 

Ob j . 1.0: The learner can determine the meaning of words. 

Ob j . l.l: Given paired sentences, one with a missing 

antonym, the learner can identify the missing 
antonym. 

Ob j . 1.2: Given paired sentences, one with a missing 

synonym, the learner can identify the missing 
synonym. 

Obj. 1.3: The learner can identify the missing homonym in 
an incomplete sentence « 

In grade 1, only antonyms (1.1) were tested. Of the first 
grade objectives tested, it was the easiest among those requiring 
interaction with print. Synonyms (1.2) were tested in grade 2. 
In grade 2 antonym items were on average easier than synonym 
items. At this level students may have been answering items by 
using memorized associations rather than by using critical 
elements of the word^s meaning to reason an answer. 

In grades 3 through 5 two additional word meaning objectives 
were tested. These were homonyms (1.3) and the target objective 
(1.0). The target objective was tested with contextual word 
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Figure 6. Relative difficulty cunong vocabulary knowledge objectives. 
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meaning, structural word meaning, sight recognition, and multiple 
meaning items. The homorym objective may have been tested incor- 
rectly* Theoretically there is a difference between homonyms, 
homophones, and homographs, (Homonyms are words wil" the same 
pronunciation and spelling but with different meanings Ci,e,, 
multiple meanings]. Homophones are words with the same pronun- 
ciation but with different spellings and meanings Ce,g,, there, 
theirl. Homographs are words with the same spelling but witii 
different pronunciations and meanings Ce,g,, The wi.nd blew, Wmd 
the watch.] > All three word categories are sometimes grouped 
together, albeit incorrectly, as homonyms. At any rate homonyms 
were tested under the target objective as multiple meaning wov^ds 
and homophones were tested under the homonym abjective. 

The homonym objective was the easiest of the word meaning 
objectives at grades 3, A, and 5, Among the homonym items, 
difficulty correlated -.49 (p < .01) with the frequency of occur- 
rence in print of the correct response. This means that as the 
frequency of the occurrence in print of the correct responses 
decreased the items increased in difficulty. <**Frequency of 
occurrence in print** refers to the number of times a word appears 
in generally read materials such as newspapers, magazines, and 
books. For example, "^the** has a high frequency of occurrence in 
print. Frequency meai.ures were taken from Kucera and Francis, 
Computational Ana lysis of P rese nt-Dav American Engl ish . > 

In grades 3 through 5 children appeared to have the most 
difficulty with the structural word meaning and contextual word 
meaning items used to test the target objective. Multiple mean- 
ing items were difficult when the word was used in an unusual way 
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or when the context was not '*rich." Generally sight recognition 
items were the easiest. These most likely represented a decoding 
task. That is, once lexical access was achieved through phonolo- 
gical recoding the meaning was readily available. 

Domain 6: Relationships (Class) 

Obj. 6,0: Thr learner can group words according to 
rexat ionships • 

Obj. 6.1: T.*e learner can group pictures into categories. 

Obj. 6.0: The learner can arrange words into given 
categories . 

Objective 6.1 was tested in grade 1 by having the students 
choose, from a group of four pictures, the one that did not 
belong. Items testing this objective were very easy relative to 
other items measuring vocabulary knowledge. There are at least 
two reasons why this could be the case. First, pictures made the 
task very concrete. Second, the use of pictures eliminated the 
necessity to decode any words. The most difficult items in this 
group were those in which the odd picture was frequently asso- 
ciated with one member of the related trio. For example, base- 
ball and bat are frequently associated together, however the 
common, critical element that linked baseball to the other pic- 
tures was not the element of meaning that linked baseball to bat. 

-Objective 6.2 was tested in second grade. Presented with a 
set of four related words and the question "What else belongs?" 
students chose, from the three answer choices, another word that 
would fit in the category. These items were generally easy. 
Variation in difficulty seemed to be related to specificity of 




categories; items representing more general categories, (e.g., 
animal) usually were easier. 

Objective 6.0 was tested in grades 3, 4, and 5 in a manner 
similar to the way objective 6.1 was tested. For this objective 
students had to choose the word that did not belong. In grade 3 
difficulty again seemed to be related to specificity of cate- 
gories. For example, deciding "beaver" did not belong with names 
of birds was harder than deciding ruler did not belong among 
names for money. In grades 4 and 5 iteus were difficult when the 
relationship was synonyms for the same concept. 



Domain 7: Relationships (Analogies) 

Obj. 7.1 The learner can determine analogous relationships 
using related pictures. 

Obj. 7.2 The learner can determine analogous relationships 
using related sentences. 

Objective 7.1 was tested in grades 2 and 3 and objective 7.2 
in grades 4 and 5. The testing formats were parallel; 7.1 used 
pictures to express the analogy, 7.2 used words. The student had 
to draw an analogy when given one pair and one member of the 
analogous pair. Students had difficulty with 7.1 analogies when 
incorrect answer choices were related to the given member of the 
second pair, but not in a way that would complete the analogy. 
This suggests that children were having difficulty determining 
the critical element of meaning which formed the relationship 
between the two pairs of words. That is, their vocabulary 
knowledge may not have been flexible and/or precise enough for 
then to sor'w out the most important elements of meaning. 

No clear patterns of difficulty emerged among the items 




testing objective 7.2. The grade 4 data suggested that "worker/ 
article created** and ••user/tool" relationships were difficult, 
however the connection was not uniformly robust. One might 
expect item difficulty to be related to the grade level (as 
determined by the SC Word List) of the words used in the analogy 
items. However, no correlations between difficulty and grade 
level were significant <p > .05). Thus, knowledge of word mean- 
ings appears to be idiosyncratic, perhaps developed through each 
individual's unique experiences with language. Furthermore, 
words at higher grade levels (as determined by the SC Word List) 
may not be inherently more difficult to learn. The data for 
objective 7.2 do support the growing body of research showing 
that vocabulary knowledge is mostly developed through reading, 
given the assumption that the greater part of an individual's 
reading experiences are unique. 

Recommendations: Vocabulary Knowledge is perhaps one of the 
most important content areas in a reading curriculum. There are 
now a growing amount of data from recent reading research to 
support a causal connection between knowledge of word mec^nings 
and variations in reading comprehension ability. The notion of a 
limited capacity processor further underscores how crucial is 
this domain. (Limited capacity processor implies that when cer- 
tain components of the reading task, i.e., decoding, access to 
word meaning, become automatic the reader can devote conscious 
attention to the meaning of the connected text.) Ironically Word 
Meaning is probably one of the most difficult domains in which to 




instruct. The evidence to support the notion that the bulk of 
vocabulary growth occurs inductively through the learning of 
meanings as words are encountered during reading is strong. 
There is an excellent article in Reading Research Quarterly. Vol. 
XXI. No. 4 (Keith E. Stanovich, Matthew effects in reading: Some 
consequences of individual differences in the acquisition of 
literacy.) that discusses the relationship of vocabulary know- 
ledae and its development to overal 1 reading achievement . I f 
this article, or the information contained therein, were 
generally available to teachers, it could help them form a theo- 
retical foundation upon which to base their reading instruction. 

The difficulty of the structural and contextual analysis, 
and multiple meaning items suggests that both precision and flexi 
bility play a role in acquiring knowledge of word meanings. 
Being able to focus on critical elements- of. meaning^ requires 
precise knowledge about a word's meaning. Working with multiple 
meanings and the connotations of words requires flexibility. 
Instruction which focuses on activities emphasizing the critical 
elements of a word's meaning should help accomplish these ends. 
Activities such as semantic feature analysis and semantic net- 
working tanderscore critical elements. Semantic networking acti- 
vities may be especially beneficial because the scheme of a 
semantic network illustrates relationships wordf? can share (e.g., 
c lass , property, example > . In addi t ion , students should be en- 
couraged to verbalize their understandings of a word's meaning, 
perhaps in situations where they tell why a particular word is a 
synonym or antonym of another word. At this level teachers may 
need to develop the concept of multiple meanings, as well as the 
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multiple meanings of specific words. 

The easiness of the homophone items relative to the other 
word meaning items suggests that their real difficulty may be 
with written production rather than with recognition and lexical 
access during reading. The corrected test method of spelling 
practice may be very useful for helping students acquire word- 
specific information necessary to spel 1 various homophones. The 
relationship to frequency suggests that children will learn to 
read most high frequency homophones indirectly as a by-product of 
reading/wr i t ing activities. 

Finally, given recent evidence of how vocabulary growth occurs 
teachers should be encouraged to provide time each day for students 
to read independently in materials of their own choosing. In 
addition, teachers should be encouraged to read daily to students 
from a variety of genre. If each of these activities were at 
some point explicitly stated in the curriculum guides, teachers 
would be less hesitant to use instructional time in their 
pursuit. 

R esearch Wj, thi.n Reach Secondary School Reading (IRA, 1987) 
contains additional instructional strategies useful for 
vocabulary development in grades 4 and 5. 



Main Idea 



Domain 5: Main Idea 

Obj. 5.8: The learner can identify the main idea of a 
reading selection* 

Obj. 5.2: The learner can identify the main idea of a story 
presented oral ly . 

Obj. 5.3: The learner can identify the restated main idea 
of a reading selection. 

Obj. 5.5: The learner can identify the paraphrased statement 
of the main idea in a reading selection. 

Obj. 5.6: The learner can identify the inferred main idea 
of a' reading selection. 

Objective 5.2 was tested in grade 1. Objectives 5.3. 5.5, 
and 5.6 were tested in grades 1 through 3. Objective 5.0 was 
tested in grades 4 and 5. Figure 7 shows the relative difficulty 
among the main idea objectives in grades 1 through 5, 

Objective 5.2 was tested by having students listen as a 
passage containing either a directly stated or implied main idea 
was read, and then choose the best statement of the main idea. 
The .^ain idea statement was either restated, paraphrased or an 
inference of an implied main idea. Objectives 5.3, 5.5, 5.6, and 
5.0 were tested by having the student read a selection and choose 
the best statement of the main idea. The reading selections 
differed in whether they contained an explicit statement of the 
main idea or whether the main idea had to be inferred. The 
correct answer choices differed in whether the main idea state- 
ment was restat-:d directly from the passage <5.3), a paraphrase 
of an explicitly stated main idea <5.5 and 5.0) or an inference 
of an implied main idea <5.6 and 5,0). 



20 

on 



more 
difficult 



Difficulty i; 



less 
difficult 



-4 



Grade 3 



Grade 2 ^' 



:: Grade 1 ♦ 



5.2 



5.3 5.5 
Objective 



Ability; 



Grade 5 * 



Grade 4 ° 



— o 




5.6 



5.0 



Figure 7, Relative difficulty eonong main idea objectives. 



In grade 1 low ability students scored at chance on items 
testing objectives 5*5 and 5.6. Also in grades 1 and 2 students 
had difficulty recognising a main idea statement that was not a 
direct restatement of the main idea sentence from the reading selec- 
tion* In grades 1 and 2, recognizing a paraphrased main idea 
statement appeared to be as difficult as recognizing an inferred 
statement of a main idea. By grade 3 recognizing restated and 
paraphrased main idea statements were of equal and less difficul- 
ty than recognizing an inferred main idea. 

Recommendations: In grades 1 and 2 instruction might be 
sequenced from restated main idea to parapharased main idea to 
inferred main idea. A differentiation between restated and para- 
phrased main ideas may not be as important in grade 3. The 
differences between restated, paraphrased, and inferred main idea 
statements need to be clearly demonstrated to students. In addition 
to locating main idea statements in reading material, activities 
could focus on having students compose main idea statements. 
Other recommendations are to vary text organization so that the 
main idea statement appears at different places (e.g., beginning, 
middle, end) in the reading selection, and to use graphic organ- 
izers to show the relationship between a main idea and the de- 
tails that support it. 

Analysis of incorrect response patterns indicated that by 
grade 5 students were not responding randomly. Instead, they 
tended to choose statements too narrow or too broad to be the main 
idea. When using multiple choice main idea practice activities, 
teachers could examine and discuss answer choices with students. 
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especially emphasizing why an answer choice is incorrect (e.g., 
too general, too specific, contradicted by other information in 
the text > , 

Finally, verbal modeling of the mental process used to 
determine a main idea should be used by teachers to instruct 
students • 
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L i bera 1 Comprehens i on 



■This content area contains the CCSD Language Arts Comprehen- 
sion domains that focus primarily on the comprehension of mate- 
rial explicitly stated in a reading selection. These domains are 
Details, Sequence, and Following Directions. Figure 8 shows the 
relative difficulty among the objectives testing literal 
comprehension in grades 1 through 5 . 

Domain 2: Details 

Obj. 2.3; The learner can identify details in a story 
presented orally. 

Obj. 2.4: The learner can identify details in a selection. 

Objective 2.3 was tested in grade 1. Students listened to 
an oral selection and answered a question about a detail 
explicitly stated in the text. Objective 2.4 was tested in 
grades 1 through 5. Studpnts read a selection and answered a 
question about a detail explicitly stated in the text. 

** There was very little variation in difficulty among the 
items testing these objectives. The data suggest that students 
have difficulty with items (obj. 2.4) requiring memory of several 
similar details (e.g., three flowers, two trees, five leaves: How 
many trees?) 

Domain 3: Sequence 

Obj. 3.2: The learner can arrange in sequential order 
illustrations of a story presented orally. 

Obj. 3.3: The learner can identify the sequence of main 
events in a reading select ion . 
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Objective 3.2 was tested in first grade. Students listened 
to a story and then selected a picture that showed what came 
first or last. Objective 3.3 was tested in grades 2 through 5. 
Students read a selection and then selected a statement that told 
what happened first or last* (In grades 3 and 4. questions also 
asked what happened just before or just after an event.) No 
clear pattern related to item difficulty emerged. 

Domain 4: Following Directions 

Obj. 4.1: The learner can complete a task after hearing a 
sot of directions* 

Obj. 4.2: The learner can complete a task after reading a 
set of directions. 

Obj. 4.3: The learner can identify steps to complete a 
given ^ask. 

Objective 4.1 was tested in grades 1 and 2. Students listened 
as directions were read and then chose a picture that showed the 
directions were followed correctly. Directions in grade 1 had 
one step. Grade 2 directions had two to three steps. Objective 
4.2 was tested in grades 3 through 5. Students read a selection 
and then chose a picture that showed the directions were followed 
correctly. The number of steps in the directions continued to 
increase from grades 3 to 5. Objective 4.3 was tested in grade 
5. Students read a passage and chose a statement which was 
correct based upon the directions in the passage. 

Item difficulty in this domain was strongly related to the 
complexity of the directions in the passage. The correlation 
between item difficulty and the number of steps in the directions 
was .77 (p = -0001 ) . 




Recommendations J Figure 8 shows the relative difficulty 
among the objectives testing literal comprehension in grades 1 
through 5. The data suggest degrees of instructional emphasis 
that should be placed on particular objectives at the various 
grade levels. In grade 1 details <2.4) is one of the hardest 
objectives in this content area, but that difficulty drops off at 
grade 2. Teachers could consider spending less instructional 
time on details (2.4) in grade 2, and placing gre^ter instruc-- 
tional emphasis on other literal comprehension skills. At grade 
3 the difficulty of the details skill (2.4) again increases, 
perhaps reflecting the expansion in the breadth of the text 
children are beginning to encounter. The instructional emphasis 
on this objective should again increase, with a focus on distin-- 
guishing between relevant and irrelevant details. By grades 4 
anu 5 the difficulty of details (2.4) again drops off. Instruc- 
tional emphasis should reflect this drop and shift in grades 4 
and 5 from the Details domain to Sequence and Following Directions. 

The difference in difficulty level between following oral 
directions (4.2) in grades 1 and 2 and other objectives is also 
striking. Consideration should be given to incorporating this 
objective into the kindergarten curriculum. 

Teachers may want to have children practice imaging a process 
help them deal with complicated directions in written text. 
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Infer en tial C ompr ehen s i on 

This content area contains the CCSD Language Arts 
Comprehension domains that focus on the comprehension of 
information inferred by a reading selection. These domains are 
Compare/Contrast, Cause/Effect, Drawing Conclusions, and 
Predicting Outcomes* Figure 9 shows the relative difficulty 
among the objectives testing inferential comprehension in grades 
1 through 5. 

Domain 8: Compare /Contrast 

Ob j . 8-0: The learner can compare and contrast elements in 
a reading selec t i on . 

Ob j . 8-1: The learner can compare and contrast details of 
pictures. 

Obj. 8-3: The learner can identify characteristics that 

compare and contrast objectives, characters, time 
and places in a reading selection. 

Objective 8.1 was tested in first grade. Students answered 

a question about a set of three pictures (e.g. , Which frog is 

<superlative form of ad jec t ive>? > . Objective 8.3 was tested in 

grades 2 and 3. Objective 8.0 was tested in grades 4 and 5. For 

objective 8.3 students responded to a question by choosing a fact 

to support a conclusion. Questions for objective 8.0 required 

the students to draw a conclusion. The former was a recognition 

task, while the latter required recall of information to make a 

decision. Quantitative variations in the item stems did not 

appear to account for the range of difficulty among the items. 

Varirtions in the content of the items suggest that difficulty 

may be related to student familiarity with the topic, although 
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Figure 9, Relative difficulty among inferential comprehension objectives. 



this cannot be empirically supported since no measure of topic fami- 
liarity is available. 

In grade 2, items in which the correct answer choice was a 
restatement of a detail tended to be easier than items in which 
the correct answer choice was a paraphrase of a detail. This 
distinction did not appear as striking in the grade 3 data* 
These results are consistent with the shift in relative 
difficulty level from grade 2 to grade 3 of the restated and 
paraphrased main idea items. 

Domain 9: Causa/Effect 

Obj. 9.1: The learner can identify cause and effect 
relationships by matching pictures. 

Obj. 9.3: The learner can identify statements that imply 
cause and effect relationships in e paragraph. 

Obj. 9.&r The learner can identify implied causal 
relationships in a reading selection. 

Objective 9.0 was tested in grade 1. Students responded to 

a set of pictures by answering a question about why something 

happened* Objective 9,3 was tested in grades 2 and 3. Objective 

9,0 was tested in grades 4 and 5. For 9.3 and 9.0 students 

responded to a reading selection by identifying the cause for an 

effect stated in a question. There was some indication that the 

d i f f icul ty level of the read ing i tems was re la ted to the amount 

of text between the sentence inferring the cause and the sentence 

telling the effect, as well as to the breadth of contextual 

constraint (i.e., the number of sentences from which the student 

had to draw information to make the inference). The degree to 

which prioT knowlc^dge had to be called upon to make the inference 
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also seemed to affect the difficulty level of the questions. 



OoMains OraMing Conclusions 

Obj. lO.as The learner can identi-Ty logical canclusions 

atmut characters or evemts described in a story 
presented orally. 

Obj. 10.3S The learner can identify logical conclusions for 
a reading selection. 

Objective 10.2 was tested in grade 1. Students answered a 
question about a story presented orally. The difficulty of items 
testing objective 10.2 was associated with the number of salient 
cues in the text. Objective 10.3 was tested in grades 2 through 
5. Students answered a question about a reading selection. In 
grade 2 the difficulty appeared to be related to how clearly 
(rather than how many) the salient cues distinguished the objec- 
tive or event being described. In grade 3 difficulty was related 
to the degree to which the conclusion could be textual ly derived. 
Those items requiring inferences from text to the reader's fund 
of knowledge about the topic (i.e., scriptually implicit) were 
more difficult than those requiring only a logical inference 
from information provided in the text (i.e., textually implicit). 
Thus when both question and response were derived from the text, 
the items were easier than when only the question was derived 
from th« text. In grade ^ most of the questions were scriptually 
implicit. Figure 9 shows that from grade 2 to grade 5 items 
became more difficult relative to student ability. In grade 2 
the student ability was two logits above the average difficulty of 
the items. In grades ^ and 5 student ability was only one logit 
above the average item difficulty. This trend continued into the 



middle grades. 



Domain: Predicting Outcomes 

Obj. 11.2: The learner can predict a logical outcome of a 
story presented orally. 

Obj, 11.3: The learner can predict a logical outcome of a 
reading selection. 

Objective 11.2 was tested in grade I. Students answered a 
question about a story presented oral ly . Objective 11.3 was 
tested in grades 2 through 5. Students answered a question about 
a reading selection. Patterns to explain ranges of difficulty 
among items did not emerge. The range of difficulty for 11.2 
items was 1.5 logits. With the exception of one or two outliers 
at each grade, the items testing objective 11.3 tended to cluster 
very close together (range .5 logits) in difficulty. 

Recommendations: The ability to answer inferential 
comprehension questions is related to general problem solving 
ability. Readers must elaborate by drawing upon past experiences 
to supply missing information. Teachers may want to model 
probLeg^-ao Iving straj^agies that can be used to make inferences 
when reading. The Rea ding Teacher (Nov., 19C3, p. 144) describes 
a strategy for improving ability to answer inferential questions. 

Certain words cue compare/contrast and cause/effect 
situations. Students should be taught to recognize the verbal 
signals of these particular test structures. (See Research 
Within Reach Secondary School. Readina. IRA, 1987 for specific 
instructional strategies.) Activities in which the teacher sup- 
plies the propositions and has the student supply the appropriate 

32 

47 



cue word might be employed. 

Discussions in which students are asked bo explain how they 
reached decisions, citing examples from text to support their 
logic, could help sLudents develop strategies for drawing conclu- 
sions • Teachers might also adopit a questioning format in some 
instructional situations to probe why a particular conclusion 
could or could not be drawn given the information in the text. 

Finally students need to understand that in answering ques- 
tions requiring an inference they should draw upon their own . 
experiences and prior knowledge about the topic. Group prereading 
activities designed to activate prior knowledge about a topic may 
help students gain this insight, Swaby, Teaching and Learning 
Reading contains examples of prereadinq strategies. 
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Analysis of Literature 



This content area contains the following domains: 
Figurative Language, Making Judgments, and Structural Elements, 
Figure 10 shows the relative difficulty among objectives in this 
content area. 

Domain 12: Figurative Language 

Objective 12«0: The learner can describe and analyze the 

use of figurative language in a reading 
selection. 

Objective 12,0 was tested in grades 3 through 5, The 
simile, metaphor, personification and idiom were tested. 
Students chose the best meaning for a short reading passage. n 
grade 3 simile items appeared to be the easiest. Metaphors and 
personification jttems were the most difficult. In grades 4 and 5 
personification items continued to be among the more difficult. 
Idioms were difficult when there was limited context to cue their 
meaning . 

Domain 13: Making Judgments 

Objective 13,1: The learner can identify sentences of 

fantasy or reality. 

Objective 13.2: The learner can identify statements of fact 

or opinion in a reading selection. 

Objective 13.1 was tested in grades 1 through 3. Students 

read three sentences and selected the one that was make-believe. 

Objective 13.2 was tested in grades 4 and 5. Students selected 

the fact from a set of three opinions and one fact, or the 

opinion from a set of three facts and one opinion. 
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Figv^e 20. Bfilative difficulty among analysis of literature objectives. 



Relative to student ability, objective 13.1 was the most 
difficult for grade 1 students. However, relative to the grade 
1 curriculum, objective 13.1 was among the easiest requiring 
interaction with print. Easy 13.1 items appeared to be those 
with talkinq animals, when what the animal said was set off by 
quotation marks. Items in which the make-believe element 
involved an inanimate object were also easy. 

Objective 13.2 was harder for grade 4 than grade 5 students. 
In both grades it was easier for students to identify opinions. 



Domain 14: Structural Elemonts 

Objective 14.0: The learner can identify structural 

elements in a reading selection. 

Objective 14.0 was tested in grades 1 through 5. Students 

answered three questions about one reading selection. The 

questions were about the plot, the setting, and a trait of one 

charac ter . 

In grade 1 plot questions were easy for students. Analysis of 
the response data showed that students were cued to an incorrect 
response if it contained language used in the stimulus. For 
example, the phrase, •*moved as fast as he could," cued students to 
the incorrect answer '•He does not move very fast." This behavior 
was especially apparent in grades 2 through 4. 

Recommendationct Fables might be useful for teaching the 
difference between fantasy and reality, particularly if students 
discussed what were the real and what were the fanciful elements- 
Teachers should teach the structure of an opinion statement 



versus that of a fact statement, and direct attention to words that 
cue opinions (e.g., should, must, ought). 

Use of story maps would help children develop concepts about 
the elements of a story. Chi ldren_;^s Literature in the Read ing 
Pro gra m (IRA, 1987) is a good resource for other activities to 
develop children's story schemata- 

I 
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